Apparatus and method for providing haptic control signal

ABSTRACT

An apparatus and a method for providing a haptic control signal of a haptic device are disclosed. The apparatus for providing a haptic control signal of a haptic device comprises: a haptic pattern data determination unit for determining haptic pattern data on the basis of at least one of an audio signal and an additional effect signal; a haptic control signal generation unit for generating a haptic control signal for controlling a vibration operation of the haptic device on the basis of the haptic pattern data; and a transmission unit for transmitting the haptic control signal to the haptic device.

TECHNICAL FIELD

The following description relates to a method and apparatus forproviding a haptic control signal.

BACKGROUND ART

As the cultural industry develops, types of content entertainingconsumers such as a concert and a video game are diversified. Consumerdemand for a method of providing visually immersive content such as avirtual reality (VR) and a three-dimensional (3D) image is increasing.In addition, consumer demand for a more tactile and realistic experienceis also increasing. Based upon such a trend, interest in a haptic devicethat provides a haptic feedback to a consumer is gradually increasing.The haptic device may adjust a haptic stimulus to be provided by thehaptic device by adjusting an amount of current flowing in the hapticdevice and a magnitude of voltage applied to the haptic device. In orderto provide a higher level of immersion for content provided with thehaptic device, it is required to develop technology for deliveringvarious tactile sensations such as rubbing, tightening, hitting,pressing, and beating.

DISCLOSURE OF INVENTION Technical Solutions

According to an aspect, there is provided an apparatus for providing ahaptic control signal, the apparatus including a haptic pattern datadeterminer configured to determine haptic pattern data based on at leastone of an audio signal and an additional effect signal, a haptic controlsignal generator configured to generate a haptic control signal forcontrolling a vibration operation of a haptic device based on the hapticpattern data, and a transmitter configured to transmit the hapticcontrol signal to the haptic device.

The haptic pattern data determiner may be configured to extract an audiobit pattern from the audio signal and determine the haptic pattern databased on the extracted audio bit pattern.

The haptic pattern data determiner may be configured to divide the audiosignal based on a frequency band, select a signal of at least onefrequency band including desired bit pattern information from signalsinto which the audio signal is divided based on the frequency band, andextract the audio bit pattern by performing half-wave rectification onthe selected signal of the at least one frequency band.

The haptic control signal generator may be configured to determine atleast one of a signal amplitude characteristic, a signal directioncharacteristic, and a signal state characteristic of the haptic controlsignal based on the haptic pattern data.

The haptic pattern data determiner may be configured to extract hapticpattern data corresponding to the additional effect signal from hapticpattern data stored in a database in advance.

The apparatus may further include a visualization data generatorconfigured to generate visualization data corresponding to the hapticpattern data. The transmitter may be configured to transmit thevisualization data to the haptic device.

According to another aspect, there is also provided a haptic deviceincluding a receiver configured to receive a haptic control signal froma haptic control signal providing apparatus, a haptic pattern dataextractor configured to extract haptic pattern data from the receivedhaptic control signal, an actuator configured to generate a vibration,and an actuator controller configured to generate an actuator controlsignal for controlling an operation of the actuator based on theextracted haptic pattern data.

The haptic pattern data extractor may be configured to restore thehaptic pattern data based on at least one of a signal amplitudecharacteristic, a signal direction characteristic, and a signal statecharacteristic of the haptic control signal.

The receiver may be further configured to receive visualization datacorresponding to the haptic pattern data from the haptic control signalproviding apparatus. The haptic device may further include avisualization data display configured to display characteristicinformation of the haptic pattern data based on the receivedvisualization data.

The haptic pattern data may be generated by the haptic control signalproviding apparatus based on at least one of audio data and additionaleffect data.

According to another aspect, there is also provided a method ofproviding a haptic control signal, the method including determininghaptic pattern data based on at least one of an audio signal and anadditional effect signal, generating a haptic control signal forcontrolling a vibration operation of a haptic device based on the hapticpattern data, and transmitting the haptic control signal to the hapticdevice.

The determining of the haptic pattern data may include extracting anaudio bit pattern from the audio signal and determining the hapticpattern data based on the extracted audio bit pattern.

The determining of the haptic pattern data may include extracting hapticpattern data corresponding to the additional effect signal from hapticpattern data stored in a database in advance.

The generating of the haptic control signal may include determining atleast one of a signal amplitude characteristic, a signal directioncharacteristic, and a signal state characteristic of the haptic controlsignal based on the haptic pattern data.

According to another aspect, there is also provided an actuator controlmethod performed in a haptic device, the method including receiving ahaptic control signal from a haptic control signal providing apparatus,extracting haptic pattern data from the received haptic control signal,generating an actuator control signal for controlling an operation of anactuator based on the extracted haptic pattern data, and controlling theoperation of the actuator.

According to another aspect, there is also provided a haptic deviceincluding an actuator controller configured to generate an actuatorcontrol signal for controlling an operation of an actuator based onhaptic pattern data, and an actuator configured to generate a hapticstimulus based on the generated actuator control signal.

The actuator controller may include a direct current (DC) to DC (DC-DC)converter configured to apply DC power to a circuit and a controlcircuit configured to generate a control signal based on haptic patterndata. The actuator may be configured to generate the haptic stimulusbased on the control signal.

The actuator controller may further include an adjuster configured toadjust an intensity of the generated control signal.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram illustrating a haptic stimulus providing systemaccording to an example embodiment.

FIG. 2 is a diagram illustrating a configuration of a haptic controlsignal providing apparatus according to an example embodiment.

FIG. 3A is a flowchart illustrating a method of extracting, by a firsthaptic pattern data determiner, an audio bit pattern from an audiosignal using a fast Fourier transform (FFT) according to an exampleembodiment.

FIG. 3B is a diagram illustrating an example of a first haptic patterndata determiner separating an audio signal of a frequency band includingdesired bit pattern information using an FFT according to an exampleembodiment.

FIG. 4A is a flowchart illustrating a method of extracting, by a firsthaptic pattern data determiner, extracting audio bit pattern from anaudio signal using a discrete wavelet transform (DWT) according toanother example embodiment.

FIG. 4B is a diagram illustrating a process of dividing, by a firsthaptic pattern data determiner, an audio signal based on a frequencyband through a DWT according to another example embodiment.

FIG. 5 is a diagram illustrating a configuration of a haptic deviceaccording to an example embodiment.

FIG. 6 is a diagram illustrating a configuration of an actuatorcontroller according to an example embodiment.

FIG. 7A is a diagram illustrating an example of a circuit of an actuatorcontroller according to an example embodiment.

FIG. 7B illustrates a circuit diagram of a control circuit operatingbased on a single control signal according to another exampleembodiment.

FIG. 7C illustrates an example of a circuit to generate various patternsof control signals according to another example embodiment.

FIG. 7D illustrates an example of a circuit for a haptic devicegenerating various haptic stimuli.

FIG. 8 is a flowchart illustrating an operation of a haptic controlsignal providing method performed by a haptic control signal providingapparatus according to an example embodiment.

FIG. 9 is a flowchart illustrating an operation of an actuator controlmethod performed in a haptic device according to an example embodiment.

BEST MODE FOR CARRYING OUT THE INVENTION

Reference will now be made in detail to embodiments of the presentdisclosure, examples of which are illustrated in the accompanyingdrawings. It should be understood, however, that there is no intent tolimit this disclosure to the particular example embodiments disclosed.On the contrary, example embodiments are to cover all modifications,equivalents, and alternatives falling within the scope of the exampleembodiments.

Although terms of “first,” “second,” and the like are used to explainvarious components, the components are not limited to such terms. Theseterms are used only to distinguish one component from another component.For example, a first component may be referred to as a second component,or similarly, the second component may be referred to as the firstcomponent within the scope of the present disclosure.

Throughout the specification, when an element, such as a layer, region,or substrate, is described as being “on,” “connected to,” or “coupledto” another element, it may be directly “on,” “connected to,” or“coupled to” the other element, or there may be one or more otherelements intervening therebetween. In contrast, when an element isdescribed as being “directly on,” “directly connected to,” or “directlycoupled to” another element, there can be no other elements interveningtherebetween.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting. As used herein, thesingular forms “a,” “an,” and “the,” are intended to include the pluralforms as well, unless the context clearly indicates otherwise. It willbe further understood that the terms “comprises,” “comprising,”“includes,” and/or “including,” when used herein, specify the presenceof stated features, integers, steps, operations, elements, and/orcomponents, but do not preclude the presence or addition of one or moreother features, integers, steps, operations, elements, components,and/or groups thereof.

Unless otherwise defined herein, all terms used herein includingtechnical or scientific terms have the same meanings as those generallyunderstood by one of ordinary skill in the art. Terms defined indictionaries generally used should be construed to have meaningsmatching contextual meanings in the related art and are not to beconstrued as an ideal or excessively formal meaning unless otherwisedefined herein.

When an embodiment is otherwise implemented, a function or operationspecified in a specific block may be performed differently from theflowchart. For example, two consecutive blocks may actually be executedsubstantially simultaneously, or the blocks may be reversed according torelated functions or operations.

Hereinafter, example embodiments will be described in detail withreference to the accompanying drawings. Regarding the reference numeralsassigned to the elements in the drawings, it should be noted that thesame elements will be designated by the same reference numerals. Also,in the description of example embodiments, detailed description ofwell-known related structures or functions will be omitted.

FIG. 1 is a diagram illustrating a haptic stimulus providing systemaccording to an example embodiment.

Referring to FIG. 1 , a haptic stimulus providing system may include ahaptic device 120 that delivers a haptic stimulus to a user and a hapticcontrol signal providing apparatus 110 that provides a haptic controlsignal to the haptic device 120. The haptic stimulus providing systemmay perform an operation of the haptic device 120 of the user throughthe haptic control signal providing apparatus 110, thereby providingvarious types of haptic stimuli corresponding to a bit pattern such asan audio bit pattern and the like of an external input signal to theuser. The haptic stimulus providing system may provide various hapticstimuli to the user of the haptic device 120, thereby providing a higherlevel of immersion for content to the user.

The haptic device 120 may be a device for providing a haptic stimulus tothe user through a vibration. For example, the haptic device 120 mayadjust a vibration intensity, a vibration direction, and a vibrationcycle to provide various haptic stimuli (e.g., rubbing touch, tighteningtouch, pressed touch, etc.) to the user. The haptic device 120 maycorrespond to various types of devices based on an environment in whichthe haptic device 120 is used. For example, the haptic device 120 may bea video game console in an environment in which the user plays a videogame. When the user participates in a concert of a singer, the hapticdevice 120 may be a cheering tool having a function to provide avibration based on a bit pattern of music. Also, the haptic device 120may include any wearable device including, for example, a wearable watchand a wearable band controlled in conjunction with a smartphone.

The haptic control signal providing apparatus 110 may generate a hapticcontrol signal for controlling a haptic stimulus provided by the hapticdevice 120 and transmit the generated haptic control signal to thehaptic device 120. The haptic control signal providing apparatus 110 mayreceive an input signal 130 and generate a haptic control signal basedon the received input signal 130. For example, when the input signal 130is an audio signal, the haptic control signal providing apparatus 110may generate a haptic control signal corresponding to an audio bitpattern of the audio signal. When the input signal 130 is an additionaleffect signal representing a predetermined special effect, the hapticcontrol signal providing apparatus 110 may generate a haptic controlsignal corresponding to the special effect. For example, when thespecial effect corresponds to a rain falling situation, the hapticcontrol signal providing apparatus 110 may generate a haptic controlsignal for controlling the haptic device 120 to generate a hapticstimulus such that the user feels a rain falling touch through thehaptic device 120.

FIG. 2 is a diagram illustrating a configuration of a haptic controlsignal providing apparatus according to an example embodiment.

Referring to FIG. 2 , a haptic control signal providing apparatus 200may include a haptic pattern data determiner 210 that determines hapticpattern data, a haptic control signal generator 220 that generates ahaptic control signal based on the determined haptic pattern data, avisualization data generator 230 that generates visualization datacorresponding to the haptic pattern data, a transmitter 240 thattransmits the generated haptic control signal to a haptic device, and adatabase 250.

The haptic pattern data determiner 210 may determine haptic pattern databased on at least one of an audio signal and an additional effectsignal. The haptic pattern data determiner 210 may include a firsthaptic pattern data determiner 211 that determines haptic pattern databased on an audio signal and a second haptic pattern data determiner 213that determines haptic pattern data based on an additional effectsignal.

The first haptic pattern data determiner 211 may extract an audio bitpattern from a received audio signal and determine haptic pattern databased on the extracted audio bit pattern. A process of determininghaptic pattern data from an audio signal will be described in detailwith reference to FIGS. 3A, 3B, 4A, and 4B.

The second haptic pattern data determiner 213 may determine hapticpattern data based on a received additional effect signal. Theadditional effect signal may be a signal representing a predeterminedspecial effect. The haptic pattern data corresponding to the additionaleffect signal may be previously generated and stored in the database250. The predetermined special effect may be, for example, a rainingeffect, a gun-shot effect, and the like based on provided content.Haptic pattern data for delivering a tactile feeling corresponding toeach special effect may be previously generated and stored in thedatabase 250. However, the scope of the special effect is not limited tothe example. The second haptic pattern data determiner 213 may identifythe received additional effect signal, extract haptic pattern datastored in the database 250 based on the identified additional effectsignal, and determine haptic pattern data based on the extracted hapticpattern data.

The haptic control signal generator 220 may include a data header unit221 that generates a data header including additional information to beincluded in a haptic control signal, an amplitude data generator 223that generates amplitude data of the haptic control signal, a directiondata generator 225 that generates vibrating direction data of the hapticcontrol signal, and a signal state data generator 227 that generatesdata associated with a signal state of the haptic control signal.

The data header unit 221 may generate additional information to beincluded in the haptic control signal. The additional informationgenerated by the data header unit 221 may include, for example,information on a format of the haptic control signal and information ona capacity of the haptic control signal. The amplitude data generator223 may generate amplitude data of the haptic control signal based onamplitude information of haptic pattern data. The amplitude data of thehaptic control signal may be used for determining a vibration intensityof the haptic device, thereby determining a characteristic of a hapticstimulus. The direction data generator 225 may generate vibratingdirection data of the haptic control signal based on vibrating directioninformation of the haptic pattern data. The direction data of the hapticcontrol signal may be used for determining a vibrating direction of thehaptic device, thereby determining a characteristic of the hapticstimulus. The vibrating direction may be any direction, for example, ahorizontal direction and a vertical direction. The signal state datagenerator 227 may generate data associated with a signal state of thehaptic control signal based on the haptic pattern data. For example, thedata associated with the signal state may include data on a time inwhich a signal is maintained at a maximum amplitude or data on a time inwhich a signal is maintained in an idle state (e.g., a state in which anamplitude is maintained as zero). The data on a time in which a signalis maintained at a maximum amplitude or the data on a time in which asignal is maintained in an idle state may be used for determiningvarious types of haptic stimulus pattern of the haptic device.

In another example embodiment, the haptic control signal generator 220may generate a haptic control signal based on haptic pattern datapreviously generated based on at least one of an audio signal and anadditional effect signal and stored in the database 250. The hapticcontrol signal generator 220 may generate a haptic control signal basedon haptic pattern data determined based on an audio signal received inreal time. Also, the haptic control signal generator 220 may generate ahaptic control signal based on pre-generated haptic pattern data storedin the database 220.

The visualization data generator 230 may generate visualization datacorresponding to the haptic pattern data. The visualization data mayinclude, for example, data on an emoticon to be added to a screen anddata on a character matching with a haptic stimulus pattern fordisplaying characteristic information (e.g., amplitude data, vibratingdirection data, etc.) associated with the haptic pattern data. Thevisualization data may include, for example, information on a displayinterface including information on a scheme for providing characteristicinformation, font information, and color information of a character fordisplaying the characteristic information of the haptic stimuluspattern. Also, when the haptic stimulus corresponds to an additionaleffect (e.g., raining effect), the visualization data may includeemoticon data (e.g., raindrop emoticon, umbrella emoticon, etc.) andimage data (e.g., raining background image, etc.) corresponding to theadditional effect. In another example embodiment, the visualization datamay include data for providing characteristic information (for example,amplitude information, vibrating direction information, etc.) of hapticpattern data in braille to a visually impaired user and data forproviding information on the haptic pattern data in a light emittingdiode (LED) pattern.

The transmitter 240 may transmit at least one of the haptic controlsignal generated in the haptic control signal generator 220 and thevisualization data generated in the visualization data generator 230 tothe haptic device. The transmitter 240 may transmit the haptic controlsignal or the visualization data to the haptic device through wired orwireless communication including, for example, Ethernet, Bluetooth,ZigBee, Wireless Fidelity (Wi-Fi), long-term evolution (LTE).

FIG. 3A is a flowchart illustrating a method of extracting, by a firsthaptic pattern data determiner, an audio bit pattern from an audiosignal using a fast Fourier transform (FFT) according to an exampleembodiment.

In an example embodiment, in operation 311, a first haptic pattern datadeterminer may perform segmentation on an input audio signal to performthe FFT on the audio signal. The segmentation may refer to dividing anaudio signal into any number of data streams. For example, the hapticpattern data generator may segment an audio signal into 256 or 1024 datastreams to smoothly perform the FFT on the audio signal. The number ofdata streams obtained through the segmentation is not limited to theexample. The first haptic pattern data determiner may segment an audiosignal into any number of data streams. For example, the first hapticpattern data determiner may segment an audio signal into 256 or 1024data streams to smoothly perform the FFT on the audio signal. The numberof data streams obtained through the segmentation is not limited to theexample.

In operation 312, the first haptic pattern data determiner may performthe FFT on the segmented audio signal to convert the audio signal from atime domain into a frequency domain. In operation 313, the first hapticpattern data determiner may divide the audio signal converted into thefrequency domain based on a frequency band. The first haptic patterndata determiner may divide the audio signal converted into the frequencydomain, into audio signals of a plurality of frequency bands to acquirean audio signal of a frequency band including a desired audio bitpattern.

In operation 314, the first haptic pattern data determiner may select asignal of at least one frequency band including desired bit patterninformation from the audio signals of the plurality of frequency bands.For example, the first haptic pattern data determiner may select asignal of at least one frequency band including bit pattern informationto be extracted such as bit pattern information associated with anoise-free audio signal, bit pattern information associated with aspecific musical instrument of an audio signal, and the like. Forexample, when an audio signal corresponding to noise is included in ahigh frequency band, a signal other than the high frequency band may beselected. Also, when an audio signal of a specific musical instrument isto be extracted from an audio signal including audio signals of variousmusical instruments, an audio signal corresponding to a frequency bandof the specific musical instrument may be extracted selectively. In someexample embodiments, when a plurality of frequency bands are required,audio signals of the required frequency bands may be extracted. A schemefor separating an audio signal into a plurality of frequency bands andselecting an audio signal is not limited to the foregoing example. Anaudio signal may be determined based on a required frequency band invarious ways.

In operation 315, the first haptic pattern data determiner may performan inverse fast Fourier transformation (IFFT) on the selected audiosignal. The first haptic pattern data determiner may convert theselected audio signal from the frequency domain into a time domainthrough the IFFT.

In operation 316, the first haptic pattern data determiner may smooththe audio signal obtained through the IFFT. For example, in a process ofsmoothing the audio signal, the first haptic pattern data determiner mayperform full-wave rectification on the audio signal converted into thetime domain through the IFFT, reduce a distortion occurring due to theFFT and segmentation performed on the audio signal, and perform aconvolution operation on the audio signal using a window function tosmoothly correct a radically changing area. The window function may be,for example, a Hanning function, a Hamming function, and a Kaiserfunction, but is not limited thereto. Also, when the convolutionoperation is performed in the frequency domain, a speed of theconvolution operation may be increased since a convolution operation inthe time domain is the same as a multiplication operation in thefrequency domain.

In operation 317, the first haptic pattern data determiner may extractan audio bit pattern by performing half-wave rectification on thesmoothed audio signal.

FIG. 3B is a diagram illustrating an example of a first haptic patterndata determiner separating an audio signal of a frequency band includingdesired bit pattern information using an FFT according to an exampleembodiment.

In an example embodiment, an audio signal 321 may be converted from atime domain into a frequency domain through an FFT. Through this,components of the audio signal may be separated based on a frequencyband. Signals 323 may be separated audio signals corresponding to sixdifferent frequency bands f₁, f₂, f₃, f₄, f₅, and f₆. However, a schemefor separating a frequency band is not limited thereto. The first hapticpattern data determiner may select an audio signal 325 of the frequencyband f₂ including desired bit pattern information from audio signalsseparated into a plurality of frequency bands. The first haptic patterndata determiner may convert the selected audio signal 325 from thefrequency domain into the time domain through an IFFT. The first hapticpattern data determiner may extract the desired bit pattern informationby performing smoothing and half-wave rectification on an audio signal427 converted into the time domain. The first haptic pattern datadeterminer may determine first haptic pattern data based on theextracted bit pattern information.

FIG. 4A is a flowchart illustrating a method of extracting, by a firsthaptic pattern data determiner, extracting audio bit pattern from anaudio signal using a discrete wavelet transform (DWT) according toanother example embodiment.

Referring to FIG. 4A, in operation 411, a first haptic pattern datadeterminer may perform segmentation on an input audio signal to performa discrete wavelet transform on the input audio signal. For example, thefirst haptic pattern data determiner may segment an audio signal into256 or 1024 data streams to smoothly perform the discrete wavelettransformation on the audio signal.

In operation 413, the first haptic pattern data determiner may separatethe segmented audio signal for each frequency band through the discretewavelet transformation. The first haptic pattern data determiner mayseparate the audio signal for each frequency band through the discretewavelet transform to acquire an audio signal of a frequency bandincluding a desired audio bit pattern. A method of separating an audiosignal for each frequency band through the discrete wavelet transformwill be described in detail with reference to FIG. 4B.

In operation 415, the first haptic pattern data determiner may select asignal of at least one frequency band including desired bit patterninformation from audio signals separated based on a frequency band. Forexample, the first haptic pattern data determiner may select an audiosignal of at least one frequency band including bit pattern informationto be extracted such as bit pattern information associated with anoise-free audio signal, bit pattern information associated with aspecific musical instrument of an audio signal, and the like.

In operation 417, the first haptic pattern data determiner may smooththe selected audio signal of the at least one frequency band. Forexample, in a process of smoothing the audio signal, the first hapticpattern data determiner may perform full-wave rectification on theselected audio signal, reduce a distortion occurring in the audiosignal, and perform a convolution operation on the audio signal using awindow function to smoothly correct a radically changing area. Thewindow function may be, for example, a Hanning function, a Hammingfunction, and a Kaiser function, but is not limited thereto.

In operation 419, the first haptic pattern data determiner may extractan audio bit pattern by performing half-wave rectification on thesmoothed audio signal. The first haptic pattern data determiner maydetermine haptic pattern data based on the extracted audio bit pattern.

FIG. 4B is a diagram illustrating a process of dividing, by a firsthaptic pattern data determiner, an audio signal based on a frequencyband through a DWT according to another example embodiment.

In an example embodiment, an input audio signal may be filtered by ahigh-pass filter 421. The audio signal filtered by the high-pass filter421 may be down-sampled and separated as a first band frequency signal.An audio signal filtered by a low-pass filter 422 may be down-sampled tobe used as an input audio signal for separating a second band frequencysignal. The input audio signal for separating the second band frequencysignal may be filtered by a high-pass filter 423, and down-sampled to beseparated as the second band frequency signal. An audio signal filteredby a low-pass filter 424 may be down-sampled to be used as an inputaudio signal for separating a third band frequency signal. In responseto such filtering and down-sampling processes being performed by filters425, 426, 427, and 428, the first haptic pattern data determiner maydivide an audio signal for each frequency band in a time domain. Forexample, when an audio signal is to be divided into signals of nfrequency bands, a filtering process may be performed 2n times. Thefirst haptic pattern data determiner may select an audio signal of atleast one frequency band including desired bit pattern information froma plurality of frequency band signals, and extract an audio bit patternby performing the half-wave rectification on the selected audio signal.

FIG. 5 is a diagram illustrating a configuration of a haptic deviceaccording to an example embodiment.

Referring to FIG. 5 , a haptic device 500 may include a controller 510that controls an operation of the haptic device, a receiver 520 thatreceives a haptic control signal and visualization data from a hapticcontrol signal providing apparatus, a haptic pattern data extractor 530that extracts haptic pattern data from the haptic control signal, anactuator controller 540 that generates an actuator control signal forcontrolling an operation of an actuator, actuators 551, 552, and 553that generate vibration corresponding to a haptic stimulus provided bythe haptic device, and a visualization data display 560 that displaysreceived visualization data.

The controller 510 may control operations of the haptic pattern dataextractor 530, the visualization data display 560, and a database 570.The receiver 520 may receive a haptic control signal and visualizationdata corresponding to haptic pattern data from a control signalproviding apparatus through wired or wireless communication (including,for example, Ethernet, Bluetooth, ZigBee, Wi-Fi, and LTE), and store thereceived haptic control signal and visualization data in the database570.

The haptic pattern data extractor 530 may extract haptic control datafrom the haptic control signal. The haptic pattern data extractor 530may restore, from the haptic control signal, a data header, amplitudedata, vibrating direction data, and signal state data of the hapticpattern data and extract haptic pattern data based on a restorationresult. The extracted haptic pattern data may be transmitted to theactuator controller through a communication module (not shown)

The actuator controller 540 may generate an actuator control signal forcontrolling operations of the actuators 551, 552, and 553 based on thehaptic pattern data. An operation of the actuator controller will bedescribed in detail with reference to FIG. 6 .

The actuators 551, 552, and 553 may generate a vibration based on theactuator control signal received from the actuator controller 540. Forexample, the actuators 551, 552, and 553 may generate a vibrationcorresponding to the haptic pattern data based on an actuator controlsignal in a form of current generated based on the haptic pattern data.Also, the actuators 551, 552, and 553 may be embodied in various formsbased on provided content. For example, in a case of a haptic deviceprovided in a shooting game, the actuators 551, 552, and 553 may bearranged on a vest worn by a user to generate a vibration such that theuser experiences a gun-shot effect. In this case, the actuators 551,552, and 553 may also be arranged in a shooting device to generate avibration such that the user experiences a shooting effect. A number ofthe actuators 551, 552, and 553 is not limited to the example as shownin the drawing.

The visualization data display 560 may display characteristicinformation (for example, amplitude information, vibrating directioninformation, etc.) of the haptic pattern data based on the receivedvisualization data. The visualization data may be, for example,information on an emoticon to be added to a screen and information on acharacter for displaying information on the haptic pattern data. Thedatabase 570 may store information on the haptic control signal,information on the visualization data, information on the haptic patterndata, and the like, for example.

FIG. 6 is a diagram illustrating a configuration of an actuatorcontroller according to an example embodiment.

Referring to FIG. 6 , an actuator controller 620 may include a controlcircuit 623 that generates a current based on haptic pattern data, adirect current (DC) to DC (DC-DC) converter 621 that converts a voltageof DC power applied to the actuator controller 620, and an adjuster 625that adjusts a value of the current generated in the control circuit623. The current generated in the control circuit 623 or the currenthaving the value adjusted in the adjuster 625 may correspond to anactuator control signal.

The control circuit 623 may generate a current based on DC power appliedfrom the DC-DC converter 621 and haptic pattern data received from ahaptic pattern data extractor 610, and transmit the generated current tothe adjuster 625. For example, the control circuit 623 may generate acurrent changing based on haptic pattern data through an H-bridgecircuit. The DC-DC converter 621 may adjust a voltage of the DC powerapplied to the control circuit 623.

The adjuster 625 may adjust an intensity of the current generated by thecontrol circuit 623 using a switch, apply the adjusted current toactuators 631, 633, and 635, and adjust vibrations of the actuators 631,633, and 635. For example, the adjuster 625 may include a switchingcircuit. The adjuster 625 may use a switch of the switching circuit toadjust ratios between primary and secondary windings of coils connectedto the actuators 631, 633, and 635 and adjust the current flowing in theactuators 631, 633, and 635 based on ratios between primary andsecondary windings of coils of selected coils. The adjuster 625 mayadjust a vibrating force of the actuators 631, 633, and 635 based on theadjusted current, so that the haptic device provides various hapticstimuli to a user based on the adjusted vibrations of the actuators 631,633, and 635. In another example embodiment, the adjuster 625 may adjusta magnitude of voltage applied to the actuators 631, 633, and 635through a variable resistor and adjust the intensity of current flowingin the actuators 631, 633, and 635 based on the adjusted magnitude ofvoltage.

FIG. 7A is a diagram illustrating an example of a circuit of an actuatorcontroller according to an example embodiment.

Referring to FIG. 7A, a control circuit 713 may include an H-bridgecircuit including four metal-oxide semiconductor field-effecttransistors (MOSFETs) or transistors. DC voltage of which a voltage isadjusted by a DC-DC converter 711 may be applied to the control circuit713. The control circuit 713 may generate a current changing based onhaptic pattern data and the voltage applied from the DC-DC converter711. The current generated by the control circuit 713 may be adjustedthrough a switching circuit 717. The switching circuit 717 may adjust aratio between primary and secondary windings of a coil connected to theswitching circuit 717 through switching and adjust a current to beapplied to an actuator 719 based on the adjusted ratio between primaryand secondary windings of the coil. Such scheme may easily and quicklychange and control three levels of voltage through a simple switchingcircuit, and may not require a complex pulse width modulation (PWM)signal intensity change or a PWM signal control time which is requiredin a typical method of controlling a PWM signal for voltage change. Avibration intensity and a moving direction of a vibrator of the actuator719 may be adjusted based on the adjusted current. Also, a hapticstimulus may be generated based on the adjusted vibration of theactuator 719.

FIG. 7B illustrates a circuit diagram of a control circuit operatingbased on a single control signal according to another exampleembodiment.

A circuit of FIG. 7B may be a circuit in which a DC-DC converter and atypical H-bridge circuit are connected with a logic circuit including abuffer and a NOT gate. Through a control circuit connected to the logiccircuit including the buffer and the NOT gate, an actuator control maybe performed based on a single control signal S31 720. Specifically, aDC voltage level required to drive a motor may be provided to a circuitby the DC-DC converter. The provided DC voltage level may be controlledby a PWM signal controller 700 based on information on a vibrationintensity change included in haptic pattern data.

When the control signal S31 720 is in a High state, a Low-state signalmay be output to a gate G1 721. In this case, a transistor 727 connectedto the gate G1 721 may be an On state so that a current flows therein.In addition, a High-state signal equal to a control signal 720 may beoutput to a gate G3 723. Also, a transistor 720 connected to the gate G3723 may be in an Off state, so that a current does not flow therein.When the control signal S31 720 is in a Low state, an output of a gateG2 722 may be a High-state signal. Also, the High-state signal may beoutput to a gate G4 724 on connection, so that a transistor 728 onconnection may be in the Off state. An output of a gate G5 725 connectedto an output of the gate G2 722 may be a Low-state signal. Also, atransistor 729 connected to the gate G5 725 may be in the On state, sothat a current flows therein. In other words, when the control signalS31 720 is in the High state, the transistor 727 and the transistor 728may be in the On state, so that the current flows in a forward directioni2. Also, when the control signal S31 720 is in the Low state, thetransistor 726 and the transistor 729 may be in the On state, so thatthe current flows in a reversed direction i1. In the above-describedmethod, a direction in which the current flows in the actuator may bedetermined based on the single control signal 720. In this example, anintensity of voltage applied to the actuator may also be adjusted basedon a change in intensity of the control signal S31 720 of the PWM signalcontroller 700. When a plurality of control signals is used,synchronization of the control signals may be required to control anoperation of a single actuator. When the plurality of control signals isnot synchronized, it is difficult to generate a desired vibrationpattern using an unsynchronized control signal. The above-describedmethod has an advantage of easily generating a vibration pattern since asynchronization process can be omitted by using the single controlsignal 720.

FIG. 7C illustrates an example of a circuit to generate various patternsof control signals according to another example embodiment.

Referring to FIG. 7C, a DC voltage level required for driving a motormay be provided to a circuit of FIG. 7C through a DC-DC converter. Theprovided voltage level may be controlled by a PWM signal controllerbased on information on a vibration intensity change included in hapticpattern data. An actuator 735 may be controlled based on a controlsignal S41 731 and a control signal S42 732. When the control signal S41731 is in a Low state and the control signal S42 732 is in a High state,a current may flow in a forward direction i2. In contrast, when thecontrol signal S41 731 is in the High state and the control signal S42732 is in the Low state, the current may flow in a reversed directioni1. In addition, an intensity of voltage applied to the actuator 735 maybe adjusted based on changes in intensity of the control signals 731 and732. Also, the actuator may be controlled based on the control signals731 and 732 in various types such as a sine wave, a half sine wave, asquare wave, a half square wave, and the like, for example.

FIG. 7D illustrates an example of a circuit for a haptic devicegenerating various haptic stimuli.

An actuating driver used for a linear resonant actuator (LRA) may be ahaptic driver integrated circuit, which generates a sine wave having aconstant amplitude and change only a frequency and an operational timeof the generated sine wave. A haptic device of FIG. 7D may generate asine wave, a triangular wave, a square wave, and the like through adriving signal generator 740, digital gain controllers 751 and 752including digital gain resistors, and OP AMPs 761 and 762. The hapticdevice may change an operational time, a frequency, and an amplitude ofeach of the generated wave, thereby generating various haptic stimuli.

Specifically, the operational time may be set in response to a digitaloutput S72 742 being output from the driving signal generator 740 in aHigh state, and an idle time may be set in response to the digitaloutput S72 742 being output in a Low state. Also, current directions i3and i4 may be determined in response to a signal associated with a phasechange being provided to an H-Bridge 753 through the driving signalgenerator 740, so that a moving direction of an actuator 770 may bedetermined based on the determined current directions. Also, the drivingsignal generator 740 may adjust an intensity of voltage provided to theactuator 770 through the digital gain controllers 751 and 752. In thiscase, an intensity of current flowing in the actuator 770 may becontrolled based on the adjusted intensity of voltage whereby a drivingintensity of the actuator 770 is determined. According to an exampleembodiment of generating various intensities of driving signal in a formof square wave based on the above-described method, the driving signalgenerator 740 may generate a square-wave pattern signal as the digitaloutput S72 742, so that a driving direction of the actuator 770 based ona current direction determined through the H-Bridge 753. The drivingsignal generator 740 may determine the driving intensity of the actuatorusing the digital gain controllers 751 and 752, thereby generating asquare-wave driving signal. According to another example embodiment ofgenerating a sine-wave driving signal, a total length of time in which asine wave is output to the digital output S72 742 through the drivingsignal generator 740 may be determined. In this example, in the drivingsignal generator 740, gains of the digital gain controllers 751 and 753may be adjusted based on a signal S71 741 and a signal S73 743. Throughthis, a form of an output signal may be adjusted based on a magnitudechange of the output signal such that output signals of the OP-AMPs 761and 762 are in a form of half sine wave. Also, the output signalsadjusted to the half sine wave may be re-output in a current directionchanged through the H-Bridge 753, so that a driving signal in a fullysine-wave form is generated. The actuator 770 may be driven based on thegenerated sine-wave driving signal. By controlling an amplitude and aphase of a driving signal in a manner similar to the scheme ofgenerating the sine-wave driving signal, a half sine eave, a squarewave, a half square wave, a triangular wave, and the like may begenerated. A form of a generated driving signal is not limited to theforegoing examples. Through an adjustment of an amplitude and a phase,any form of driving signal may be generated. The driving signal maycorrespond to an actuator control signal.

FIG. 8 is a flowchart illustrating an operation of a haptic controlsignal providing method performed by a haptic control signal providingapparatus according to an example embodiment.

Referring to FIG. 8 , in operation 810, a haptic control signalproviding apparatus may determine haptic pattern data based on at leastone of an audio signal and an additional effect signal. The hapticcontrol signal providing apparatus may extract an audio bit pattern fromthe audio signal and determine the haptic pattern data based on theextracted audio bit pattern. For example, the haptic control signalproviding apparatus may divide a frequency band of an audio signalthrough an FFT or wavelet transform, select a signal of a frequency bandincluding desired bit pattern information, extract an audio bit patternfrom the selected signal, and determine haptic pattern data based on theextracted audio bit pattern. The haptic control signal providingapparatus may extract haptic pattern data corresponding to an additionaleffect and stored in a database in advance, and determine haptic patterndata based on the extracted haptic pattern data.

In operation 820, the haptic control signal providing apparatus maygenerate a haptic control signal based on the haptic pattern data. Forexample, the haptic control signal providing apparatus may determine atleast one of a signal amplitude characteristic, a signal directioncharacteristic, and a signal state characteristic of the haptic controlsignal based on the haptic pattern data.

In operation 830, the haptic control signal providing apparatus maytransmit the haptic control signal to a haptic device. The hapticcontrol signal providing apparatus may transmit the haptic controlsignal to the haptic device through wired or wireless communicationincluding, for example, Ethernet, Bluetooth, ZigBee, Wi-Fi, and LTE.

FIG. 9 is a flowchart illustrating an operation of an actuator controlmethod performed in a haptic device according to an example embodiment.

Referring to FIG. 9 , in operation 910, a haptic device may receive ahaptic control signal from a haptic control signal providing apparatus.In operation 920, the haptic device may extract haptic pattern data fromthe received haptic control signal. For example, the haptic device mayrestore the haptic pattern data based on at least one of a signalamplitude characteristic, a signal direction characteristic, and asignal state characteristic of the haptic control signal.

In operation 930, the haptic device may generate an actuator controlsignal based on the extracted haptic pattern data. For example, thehaptic device may generate an actuator control signal in a form ofcurrent based on the haptic pattern data through an H-bridge circuit,and adjust the actuator control signal by adjusting a ratio betweenprimary and secondary windings of a coil through a switching circuit. Inoperation 940, the haptic device may control an operation of an actuatorbased on the actuator control signal. For example, the actuator maygenerate a haptic stimulus by generating a vibration corresponding tothe haptic pattern data based on the actuator control signal.

The components described in the exemplary embodiments of the presentinvention may be achieved by hardware components including at least oneDigital Signal Processor (DSP), a processor, a controller, anApplication Specific Integrated Circuit (ASIC), a programmable logicelement such as a Field Programmable Gate Array (FPGA), other electronicdevices, and combinations thereof. At least some of the functions or theprocesses described in the exemplary embodiments of the presentinvention may be achieved by software, and the software may be recordedon a recording medium. The components, the functions, and the processesdescribed in the exemplary embodiments of the present invention may beachieved by a combination of hardware and software.

The methods according to the above-described example embodiments may berecorded in non-transitory computer-readable media including programinstructions to implement various operations of the above-describedexample embodiments. The media may also include, alone or in combinationwith the program instructions, data files, data structures, and thelike. The program instructions recorded on the media may be thosespecially designed and constructed for the purposes of exampleembodiments, or they may be of the kind well-known and available tothose having skill in the computer software arts. Examples ofnon-transitory computer-readable media include magnetic media such ashard disks, floppy disks, and magnetic tape; optical media such asCD-ROM discs, DVDs, and/or Blue-ray discs; magneto-optical media such asoptical discs; and hardware devices that are specially configured tostore and perform program instructions, such as read-only memory (ROM),random access memory (RAM), flash memory (e.g., USB flash drives, memorycards, memory sticks, etc.), and the like. Examples of programinstructions include both machine code, such as produced by a compiler,and files containing higher level code that may be executed by thecomputer using an interpreter. The above-described devices may beconfigured to act as one or more software modules in order to performthe operations of the above-described example embodiments, or viceversa.

A number of example embodiments have been described above. Nevertheless,it should be understood that various modifications may be made to theseexample embodiments. For example, suitable results may be achieved ifthe described techniques are performed in a different order and/or ifcomponents in a described system, architecture, device, or circuit arecombined in a different manner and/or replaced or supplemented by othercomponents or their equivalents. Accordingly, other implementations arewithin the scope of the following claims.

The invention claimed is:
 1. An apparatus for providing a haptic control signal, the apparatus comprising: a haptic pattern data determiner configured to determine haptic pattern data based on an audio signal and a predetermined special effect signal; a haptic control signal generator configured to generate a haptic control signal for controlling a vibration operation of a haptic device based on the haptic pattern data; and a transmitter configured to transmit the haptic control signal to the haptic device, wherein the haptic pattern data determiner is configured to extract haptic pattern data corresponding to the predetermined special effect signal from haptic pattern data stored in a database in advance, and the haptic control signal comprises: a data header including at least one of information on a format of the haptic control signal and information on a capacity of the haptic control signal; an amplitude data for determining a vibration intensity of the haptic device; and a signal state data including data for a time when a vibration of the haptic device maintains a maximum amplitude or data for a time when a vibration of the haptic device maintains an idle state.
 2. The apparatus of claim 1, wherein the haptic pattern data determiner is configured to extract an audio bit pattern from the audio signal and determine the haptic pattern data based on the extracted audio bit pattern.
 3. The apparatus of claim 2, wherein the haptic pattern data determiner is configured to divide the audio signal based on a frequency band, select a signal of at least one frequency band including desired bit pattern information from signals into which the audio signal is divided based on the frequency band, and extract the audio bit pattern by performing half-wave rectification on the selected signal of the at least one frequency band.
 4. The apparatus of claim 1, wherein the haptic control signal generator is configured to determine at least one of a signal amplitude characteristic, a signal direction characteristic, and a signal state characteristic of the haptic control signal based on the haptic pattern data.
 5. The apparatus of claim 1, further comprising: a visualization data generator configured to generate visualization data corresponding to the haptic pattern data, wherein the transmitter is configured to transmit the visualization data to the haptic device.
 6. The apparatus of claim 1, wherein the haptic device is configured to receive the haptic control signal from the apparatus and generate an actuator control signal for controlling an operation of an actuator based on the received haptic control signal.
 7. A haptic device comprising: a receiver configured to receive the haptic control signal from the haptic control signal providing apparatus of claim 1, a haptic pattern data extractor configured to extract haptic pattern data from the received haptic control signal; an actuator configured to generate a vibration; and an actuator controller configured to generate an actuator control signal for controlling an operation of the actuator based on the extracted haptic pattern data.
 8. The haptic device of claim 7, wherein the haptic pattern data extractor is configured to restore the haptic pattern data based on at least one of a signal amplitude characteristic, a signal direction characteristic, and a signal state characteristic of the haptic control signal.
 9. The haptic device of claim 7, wherein the receiver is further configured to receive visualization data corresponding to the haptic pattern data from the haptic control signal providing apparatus, and the haptic device further comprises: a visualization data display configured to display characteristic information of the haptic pattern data based on the received visualization data.
 10. The haptic device of claim 7, wherein the haptic pattern data is generated by the haptic control signal providing apparatus based on audio data and predetermined special effect data.
 11. A haptic device comprising: an actuator controller configured to receive the haptic control signal from the haptic control signal providing apparatus of claim 1, to extract haptic pattern data from the received haptic control signal, and to generate an actuator control signal for controlling an operation of an actuator based on the haptic pattern data; and an actuator configured to generate a haptic stimulus based on the generated actuator control signal.
 12. The haptic device of claim 11, wherein the actuator controller comprises: a direct current (DC) to DC (DC-DC) converter configured to apply DC power to a circuit; and a control circuit configured to generate an actuator control signal based on haptic pattern data, wherein the actuator is configured to generate the haptic stimulus based on the control signal.
 13. The haptic device of claim 12, wherein the control circuit is configured to generate the control signal based on a bridge circuit.
 14. The haptic device of claim 12, wherein the actuator controller further comprises: an adjuster configured to adjust an intensity of the generated control signal.
 15. The haptic device of claim 14, wherein the adjuster is configured to adjust the intensity of the control signal by adjusting a turns ratio of a coil connected to the actuator through a switch.
 16. The haptic device of claim 14, wherein the adjuster is configured to adjust the intensity of the control signal through a variable resistor.
 17. The haptic device of claim 14, wherein the actuator controller is configured to generate an actuator control signal that adjusts a form of the control signal by adjusting an amplitude and a phase of the control signal.
 18. The haptic device of claim 1, wherein the predetermined special effect stored in the database in advance includes at least one of a raining effect and a gunshot effect. 